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ABSTRACT 

the steps taken by a large urban school district to 
develop and iirplement an objectives-based curriculum with 
criterion -referenced assessment of student progress are described. 
These steps include: goal setting^ development of curriculum 
objectives^ construction of assessment exercises, matrix saorpling in 
test administration, and reporting of results. The model provides for 
local control of educational objectives with credible accountability 
to the people for instructional results. (CK) 



FILMED FROM BEST AVAILABLE COPY 



APPLICATION 
OP 

THE NATIONAL ASSESSMENT OP EDUCATIONAL PROGRESS PHILOSOPHY 
IN SAN BERNARDINO CITY UNIFIED SOiOOL DISTRICT 

Lewis A* Bonnex 

BACKGROUND 



This paper describes the steps taken by a large urban school district to 
develop and implement an objectiTes*based curriculum with criterion- 
referenced assessment student progress. 

The program was promoted hy a desire to help the school district be more 
responsiTt to rapidly changing student needs. It was recognized that 
student needs vary from time*to«time and pIace*to*pIace and that no 
standardized program could be expected to be effective in moving all 
students toward educational goals* Rather, it was essential to constantly 
adapt the curriculum to differert student backgrounds and learning styles* 

In order to adapt the curriculum to changing circumstances, staff members 
needed (1) the freedom to change their instructional strategies and (2) 
the information to determine which instructional strategies are effective 
in achieving objectives* This paper is specifically concerned with how 
one local school district developed a program to give staff the freedom to 
be responsible for student learning* 



RELATED PROJECTS 

The effort of City Schools to develop an objectives-based curriculum with 
criterion»referenced assessment of student progress is analogous to the 
Education Commission of the States* National Assessment of Educational 
Progress* NAEP has involved students, teachers, curriculum experts, and 
laymen in setting objectives for reading, math, social studies, science, 
and citizenship* Objectives have been stated for nine year olds, 
thirteen year olds, seventeen year olds, and adults* Exercises have been 
constrttctai to MMess students* skill in mastering the objectives* The 
exercises have been administered to a nationwide ample of students and 
adnlts, and the results have been reported for each age group in each 
area of emphasis* This information is presented to lay and professional 
groups for examination and cooments regarding appropriate educational 
prloxities* 

Similarly, the development of an obJectives*based curriculum with 
criterion*referenced assessment of student progress can give local Board 
members and community members the information necessary for setting 
educational priorities* These priorities can provide general direction 
for Instructional activities* Staff members can have the freedom to 
apply their professional training in developing and implementing instruc* 
tiottal programs within priority areas* 
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The similarities between the City Schools* aevelopmental effort and NAEP^s 
thrust led to a cooperative effort in the spring of 1972* NAEP consultants 
worked with City Schools* teachers in setting objectives and constructing 
assessment exarcises* The consultants ^vfered advice in developing a 
matrix sample to effectively and efficiently assess student progress. 
City Schools* personnel followed through by involving a variety of groups 
in commenting on educational results* The results in City Schools have 
led to the sort of educational dialogue envisioned by the Education 
Commission of the States* 



STEPS IH DEVELOPING A llESPOHSIVE CURRICULUM 



Overview 



The City Schools began the developmental process with goal^setting in the 
spring of 1971 and followed up with the establishment of curriculum 
objectives in the early spring of 1972« Exercises were constructed to assess 
student progress on selected objectives in the late spring of 1972, and 
student skills in performing the tasks specified in the exercises were 
assessed in May of 1972* A report on student mastery of curriculum 
objectives in the fall of 1973 led to the establishment of educational 
priorities for the 1973*74 school year. 

More specifically, these steps included; 

Coal Setting 



Members of the San Bernardino Board of Education launched the develop^ 
mental program in October, 1970 by scheduling a series of workshop 
sessions for the purpose of identifying goals of education* Tentative 
statements of eight educational goals ranging from, '^Students will be 
able to appreciate the fine arts'* to **Students will be able to apply the 
principlea of mathematics,** were developed and distributed to all secondary 
students, staff members, and other groups potentially interested in 
education in San Bernardino* These groups were invited to make suggestions 
regarding delations, revisions, and additions. A final workshop session 
was held to incorporate the resulting suggestions in the final statement 
of goals* These goals, summarised in Table T, were approved by the Board 
in July, 1971* 
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TABLE I 



GOALS OF EDUCATION 



I. Students will be able to 
express their ideas and 
understand the ideas of 
others. 



II. Students will be able to 
apply the principles of 
mathenatics in their every- 
day life and in accor^f^nce 
with the demands of their 
career. 



III. Stud<>nts will be able to 
appreciate the fine arts. 

IV. Stiidents will be prepared 
to enter a vocation or 
career. 



V. Students will establish 
sound health habits r 



VI. Students will develoi. and 
naintain high standards cf 
citizenship. 



VII. Students will acquire a 
fundamental *mder standing 
of the soci<kl sciences. 

VIII. Students will acquire a 
general background in the 

^ natural sciences. 



Development of Curriculum Objectives 

A Curriculum Task Force composed of twenty^ highly respected teachers on 
full released time used the eight goals as a framework for articulav:ing 
reasonable expectations of student progress in each goal area at grades 
39 69 9^ and 1.1. These teachers had been in the classroom the previous 
semester and vere considered well qualified to make Judgments regarding 
reasonable expectations for student progress. They stated curriculum 
objectives for each cell in the matrix illustrated in Table II. 
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TABLE II 
CURRICULUM 03JECTI\^ES MATRIX 



Goals 



12th 
Grade 



Conununi- 
cations 



II 



Math 



III 

Fine 
Arts 



IV 



Careers 



Health 



VI 

Citizen 
ship 



VII 

Social 
Studies 



VIII 

Natural 
Science 



9th 

<u Grade 
> 

0) 
1-3 

2 6th 
^ Grade 



3rd 
Grade 



In the goal area of language arts, the Teacher Task Force stated over 
600 objectiires. Soiie of these objectives are reproduced in Table 111. 
Similarly, in social studies the Teacher Task Force stated 200 objectives. 
Sone of these are also seen in Table III. A total of approximately 1800 
objectives gave substance to the formulae provided by the eight goals. 

An effort was made to involve as many teachers as possible in reviewing 
and revising the curriculum objectives. The Teacher Task Force visited 
faculty rooms and attended department meetings to solicit cooMents 
regarding the objectives. As might have been expected, however, this 
effort met with some coolness and, in general, did not generate much 
interest. 

Construction of Assessment Exercises 

The search for guidance in constructing instruments to measure student 
progress relative to the objectives led to association with the National 
Assessment of Educational Progress. NAEP consultants assisted the teachers 
in constructing exercises congruent with local needs. This proved to be a 
viable procedure for applying psychometric expertise to local problems. 
It was also valuable in developing staff skill in measurement. It should 
be noted, however, that, in the interest of maintaining the project as 
primarily a teacher effort, the judgments of teachers regarding practicality 
and reasonableness were weighed more heavily than psychometric sophistication. 
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TABLE III 



EXAMPLES OF 

Language Arts 



Reading 

3rd finds the toain point of a 
paragraph 

6th - differentiates between main 
point and supplementary 
details 

9th - arrives at a general prin* 

ciple after examining details 

12th *> consider source of informs* 
tion for accuracy and 
appropriateness 



,UM OBJECTIVFS 
Social Studies 



3rd - given geographic and climatic 
conditions^ pupil will illus* 
trate probable shelter and 
clothing patterns 

6th - given a list of behaviors, 
pupil will group them as 
either learned or inherited 

9th • list ways a given society 

meets people's needs through 
geographical resources 

12th - describe ways an individual 
might choose to influence 
public policy 



Literature 

3rd • describe story characters 

6th *- aware of development of a 
character 

9th *- describe character and 
setting 

12th * aware that character 

develops out of conflict 



Exercises were constructed to measure selected objectives at grades 3, 
6, 9, and 12 at each of the eight goal areas. It required 10 weeks for 
the teachers to construct thirty-two subtests for an experimental 
criterion*referenced assessment battery. This provided for assessment of 
objectives at grades 3, 6, 9, and 12 in each goal area. 

Scoring criteria were set for each item in the battery. This was 
accomplished by pilot testing potential exercises and resulted in a 
variety of modif ications, revisions and, in some cases, well advised 
elimination. The most promising items were included in the criterion* 
referenced assessment battery. 

Matrix Sampling in Test Administration 

The District's strategy involving use of a sample to estimate District- 
wide student performance also coincided with the NAEP approach to assess^ 
fluent. In this case, the NAEP consultants advised District personnel in ^ 
matrix sampling procedures. Test administration was made maximally 
efficient by sampling both students and test items. The population of 
2,000 students at each grade level was sampled to achieve 200 students 
responding to each item. The sample of eleoMntary ^/upils was chosen by 
randomly selecting Elementtxry Schools for inclusion in the project and 
then randomly selecting pupils at these schools. All secondary schools 
were included in the sample, with sampling of each student body^ 

The Teacher Task Force spent the latter part of May and part of June, 
1972, organising and administering the test battery. Tests were color* 
coded for each cell of the ooatrix. These tests were organized according 
to the matrix saoq)ling design and collected into batteries for each grade 
level involved in the assessment. 

The actual administration of the test battery involved advising principals 
of their possible involvement in the project and arranging for the most 
convenient testing times and places. The testing time for elementary 
pupils was one hour and the testing time for secondary pupils was two hours. 
This was accomplished with relative ease at the elementary and Junior 
high school levels; however, a lesson was learned regarding the avail- 
ability of high school seniors in May of their graduation year. 

The Teacher Task Force administered and scored the tests in June of 1972. 
This led to added discoveries regarding both the practicality and ease of 
administering exercises that required special tape recordings of student 
responses. These exercises were much more difficult to administer and 
score than the stand '^rd paper-and-pencil exercises but the information 
gained was unusually helpful for describing student skills in affective 
areas. 

Reporting of Results 

The Teacher Task Force tabulated results in terms of percent of students 
meeting an established criterion for each item. It was possible, for 
example, to report the percent of third grade students who could add 
i^ole numbers or read a thermometer. Similarily, it was possible to 
report the percent of ninth graders who could write a school announcement 
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at least as good as a selected sanple of student work. (See exai^le in 
Figure 1.) 



Figure I 

COMMUNICATIONS - FUNCTIONAL WRITING 

Interral Ninth Gradie 

CONCEPT: The aechanics of writing are functional to coowmication* 

WRITING ANNOONCEMlMyS 

Directions: Write an announcement for the school bulletin advertising a 
aoTie to raisa ttoney for student body funds. Be sure and include all 
necessary infomation to aake the announcement clearly understood. In 
addition, your paper will be graded according to the following criteria: 

1. Punctuation 

2. Capitalization 

3. Sentence structure 

SAMPLE RESPONSES 

^'Acceptable" Response: 

Thirty percent of the papers were similar to the above response* 



Note: Fourteen percent of the students surveyed made no response* One 
percent of the studants responded with **don*t know*" 



SAMPLE RESPONSES 



"Unacceptable" Response: 




"' -nr ^ 



Thirty percent of the responses Iti the "unacceptable" category were 
conqparable to the above response. 



"Fair" Response; 



Twent7«*fiTe percent of the responses in the "fair" category were 
coflq;>arable to the above response. 
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A coaprchenilTe report w«i prepared during Jane, 1972, deUlling third, 
sixth, ninth, and twelfth grade student perfonunce In each goal area. 
The report wai organized Into nine chapters, one reporting student perfor* 
mance In each goal area, and one with teacher ceoMnti regarding the project. 



IMPLICATIONS OF THE PROJECT 

In accordance with the NAEP'» philosophy, the District distributed copies 
of the report to Board Members, students, teacher«, and parent groups for 
coMents and suggestions regarding both the pilot project and iflq)Ilcations 
for Distrlctiride instructional priorities. Th« report was presented ».o 
Board Meabers in January, 1973. 

The Board Members responded by coBMenting on the coMioasense nature of the 
objectives. They also gained Insight regarding the distinction bet. <ien 
noni>ref.irenced and crlterion>referenced reporting of student progress* 
It becaae apparent thai both types of reports werii iitportsnt to Board 
Members. 

In a following neetlng the Trustees Initiated « discussion of instructional 
priorities. They considered nons-referenced test data and criterion- 
referenced test data. They expressed concern with scores on nom-referenced 
test data, but in considering their responsibility for setting instruc- 
tional priorities, their attention focused on the Teacher Task Force reports. 
They discussed the data and, in a subsequent Meting, settled on the four 
C 8 - coaprehenslon, co^>ositlon, cooputation, and careers - as instruc- 
tional priorities for the 1973-74 school year. 

The administration was asked to work with staff aeabers in establishing 
reasonable standards of pupil progress in each of these priority «reas. 
Staff menbers were to use the existing curriculum objectives as a starting 
point for this objective setting process. Also, staff members were to 
prepare exercises for assessing student mastery of objectives. 

The Trustees formally approved the folloving resolution: 

"BE IT RESOLVED that this Board of Education adopt a career 
concept of education as a new thrust of edacation in this 
school district. 

"BE IT FURTHER RESOLVED that the career concept of education 
encompassing the 5 C's be developed by the administration and 
teachers and be implemented on an individual school basis. 

"BE IT ALSO RESOLVED that the administration and teachers be 
directed to present plans for implementation with achieveaent 
expectations at grades 3, 6, 9, and 12, and appropriate assess- 
ment procedures involving norm-referenced tests, criterion- 
referenced tests and other procedures as required to reflect 
student progress in mastering objectives. 

"BE IT ALSO RESOLVED that the career concept in education should 
be carried as a thrust in adult programs as well; thereby making 
a total careers commitment to this comunity." 
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The Board Ntabtrs stated their recognition that Board Me«btrs need to be 
responsible for atttlng priori tiesr whlU staff sesbars are responsible 
for defining ihd loipleMnting the instructional program to attain desired 
results. This nould suggest that an enrlronMnt had been created in 
nblch staff Mi^rs could have the freedom to adjust their instructional 
atrategias to changing students^ needs and be responsible for students^ 
learning. 

Taking «ore of an insiders viev^ the project can be looked upon as part of 
a staff deralopMnt program designed to focus attention on Intended 
instructional results. The curriculun obJectiTes provide SKHlels for 
schools to use in developing objectives^based instruction at the school 
level. The criterion*referenced assessment battery ainilarly provides 
exsaples of assesss^nt techniques. In this sumner staff neiibers begin to 
acquire the Masurettent skills necessary for regularly collecting v^lid 
infonution rtf;arding ifhat instructional strategies are effective in 
helping students naster objectives. 

The administration has encouraged principals to work with their staff 
members in establishing school objectives congruent with District priorities. 
The responsibility has been placed on local schools to develop programs 
consistent with their students^ needs and congruent with District priorities. 
The principals have stated that working with staffs in defining appropriate 
school objectives gives them a vehicle for working cooperatively with 
teachers in improving the instructional program. 

There is an additional implication of the project which deserves attention^ 
particularly in view of the State of California's efforts to shift the 
burden of school finance from the local property tax to a combination of 
state^wide property incooM and sales taxes. One can speculate that as the 
state assumes more responsibility for school financot it will begin to set 
rather specific state-wide expectations for student progress. This has the 
potential of conflicting with a deep-seated reverence for local control of 
education. 

The San Bernardino siodel provides for local control of educational 
objectives with credible accountability to the people for instructional 
results. Local persons can establish objectives and identify locallyt 
acceptable measures of student progress. The State Department of Education 
can maintain quality control of the assessment process by acting as an 
independent educational auditor to certify to the reliability and validity 
of testing techniques. In this msaner» discrepancies between actual and 
expected student progress can be identified and become a subject for the 
scrutiny of professional and lay groups at both local and st'ite levels. 
San Bernardino City Schools' experience suggests that discussion of 
educational results provides a valuable information base for setting 
educational priorities. 
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S«a B«niardino*s txptritnct d€Mastr«tts kaw a local district cmn involve 
staff Mi^rs in dtTeloping an obJectiTea-b«atd curricalua vith critttion* 
refarcncad asstasMttt of student progress* The program has given stsff 
SMbers the freedom to be responsible for results and seems to have forged 
a more comfortable and accountable relationship between staff members and 
the Board of Education. An opportunity has been provided for translating 
c o mm un ity priorities into accountable instructional programs* 
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